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ABSTRACT 


This study examined the effects of affective student characteristics, ICT usage, and school factors on the mathematics 
achievement of students in Turkey by making use of PISA 2102 results. Multilevel analysis of variables was conducted 
since students (N = 4848). Were nested within schools (N = 170). The student affective characteristics were mathematics 
interest and mathematics self-efficacy. School level predictors were the extracurricular mathematics activities and student 
mathematics teacher ratio. Results from this study found that mathematics self-efficacy was positively correlated with 
mathematics achievement, while ICT usage was negatively correlated with the student level. For school level, an 
exploratory analysis showed that mathematics extracurricular activities at school and math teacher-student ratio could be 


the predictors of mathematics mean mathematics achievement. 
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INTRODUCTION 


Theoretical Framework 


The motivation of this study is a claim raised by Davis (1994), where he stated that the problem is not that students 
cannot learn mathematics, but they are not trying to learn. That is to say, the students do not show enough effort for 
learning, which led us to question the cognitive abilities and quality of instruction that are mostly used as the 
variables for improving student learning. For instance, Schmidt, Raizen, and McKnight (1997) investigated the US 
mathematics and science education and analyzed 491 curriculum guides and 628 textbooks from all over the world. 
They noted the problems in mathematics education according to the results of Third International Mathematics and 
Science Study (TIMSS) and focused on curriculum, textbooks, and instructional quality or teacher effectiveness, 
where no affective characteristics of students were provided as a reason for the low levels of mathematics scores in 


the TIMSS. 


The relation between learning in general and affective factors have been documented previously by 
Krathwohl, Bloom, and Masia (1984). They stated that the affective domain is mostly related to the feelings and 
emotions of individuals, more specifically students. According to their classification, affective domain can be 
partitioned into sub categories. These categories are, from simpler to more complex ones, receiving, responding, 


valuing, organization, and characterization. 
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OBJECTIVE 


Taking affective characteristics into account, as advised by Davis (1994), the goal of this study is to investigate the 
relationship between mathematics achievement and some student affective characteristics and school factors by using the 
data from Programme for International Student Assessment (PISA) 2012 results. There are two research questions for this 


study: 
e How much do schools vary in their mean mathematics achievement in Turkey? 


e Which student - and school - level variables are related to mathematics achievement of students from 7th to 12th 


grades in Turkey? 
METHOD 


In this section, the data source, participants, variables, and data analysis that was carried out in this study were described. 
Also, most of the variables had a composite structure, so, the way the researchers dealt with this situation in the data 


analysis section was described as well. 
Data Source 


PISA is an international test administered by OECD countries since 2000. The disciplines included in the study are 
mathematics, science, and reading. The aim of the test is to measure 15-year-old students' problems solving and cognition 
in daily life, where results are supposed to be used for improving education. In the 2012 version of the test that was used 
for this study, participants were 34 OECD countries and 31 partner countries, where the total number of students was 510 


000 (Organisation for Economic Co-operation and Develop & Programme for International Student Assessment, 2014). 
Participants 


The sample of this study consists of the participants of the international PISA 2012 test in Turkey. Overall, 4848 students 
took the exam in 170 different schools. The students who took the PISA 2012 test ranged from 7" to 12" grades. The 


schools can be categorized into public and private ones. 
Variables 


Since this study has a multilevel structure, that is some of the level-1 units are nested in level-2 units, there are two levels 
of variables, which are student level data and school level data. Figure | indicates the independent variables which were 


used to estimate mathematics performance on both levels. 


Figure 1: Description of Variables 


Variables 
Student Level Variables 
Math interest (INTMAT) Estimated from 4 Likert questions. Higher scores represent higher levels of 
mathematics interest. 
Mathematics Self-Efficacy Estimated from 8 Likert questions. Higher scores represent higher levels of 
(MATHEFF) mathematics SELF EFFICACY. 
Use of ICT in Mathematics Estimated from 7 questions. Higher scores represent higher levels of ICT 
Lessons (USEMATH) usage in mathematics classrooms. 
School Level Variables 
Mabemarles exfacunculat The amount of extracurricular activities at school 
activities at school (MACTIV) : 
Class size (CLSIZE) The class size 
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Math Teacher-student ratio 


(SMRATIO) Ratio of number of mathematics teachers to number of students. 


Data Analysis 


The data source and codebooks for this study are downloaded from Organization for Economic Cooperation and 
Development website (https://www.oecd.org/pisa/pisaproducts/database-cbapisa2012.htm). Since students are nested 
within schools, a multilevel structure model and student version of Hierarchical Linear Modeling (HLM 7.3) software were 


used to analyze the data. SPSS 22 software was used for the descriptive analysis. 


In order to answer the first research, question the fully unconditional model was run. In the second step math 
interest, math self-efficacy, and use of ICT in math lessons were included as level | predictors. All of the level 1 variables 
were included with fixed and random effects. Also, exploratory analysis was run in order to see the potential level 2 
variables to explain school variation both in intercepts and slopes. In final model, level | predictors with significant effects 
were kept in the model and level 2 predictors added to intercept and slopes where school variations were significant in the 


second model. 
RESULTS 
One-way Random Effects ANOVA Model 


Table 1: The Results of the Fully Unconditional Model 


Fixed Effects Coefficient (SE) t (df) p Reliability 
Model for Mean School Math Ach ( Po) 


Intercept (79 ) 443.97 (6.08) | 73.02 (150) | <.001 974 
Random Effects (Var. Components) | Variance | df v 
Var. in school means (T 9 ) 5363.43 | 149 | 7529.77 (p < .001) 
Var. within schools (o” ) 3146.83 


The result of the fully unconditional model shows that the average school mean mathematics achievement (7 99 ) 


is 443.97 and significantly different from zero (i.e., p < .001). The within school mathematics achievement variation (a7) 
is 3146.83. The between school mathematics achievement variation is (Tg9) is 5363.43 and significantly different from 
zero (1.e., p < .001) which means that the mathematics achievement differs across schools. The reliability coefficient for the 
intercept is .97 which suggest that about 97% of the variation in the means of mathematics achievement is explained by the 
true mathematics achievement variation between groups. The intra-class correlation (ICC) which is calculated using the 
formula in Equation | suggests that about 63% of the variation is students’ mathematics scores are accounted by school 


level variation. 


between school variance (T99) 
ICC = 


within school variance (o2)+between school variance (T99) 


Following the fully unconditional model, random coefficients model was conducted to investigate the significant 


level-1 predictors for the mathematics achievement. Math interest (INTMAT), Mathematics Self-Efficacy (MATHEFF), 
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Use of ICT in Mathematics Lessons (USEMATH) were group centered and added to the model. Intercept and slopes are 


allowed to vary randomly. 


Table 2: The Results of the Random Coefficient Models/ Random Coefficients Model (group mean centering) 


Fixed Effects Coefficient (SE) t (df) p Reliability 

Model for Mean School Math Ach (69) 
Intercept (Vo9) 444.80 (6.07) 73.29 (149) <.001 973 
Model for INTMAT slope (6; ) 
Intercept (719) 92 (1.29) .71 (149) 476 .176 
Model for MATHEFF slope (f,) 
Intercept (V29) 20.90 (1.78) 11.75 (51) <.001 167 
Model for USEMATH slope(63) 
Intercept (V9) -8.57 (1.13) -7.57(149) <.001 146 

Random Effects (Var. Components) | Variance | df V a 

Var. in school means (Tq) 5288.39 | 145 | 6014.13 (p< .001) 

Var. in INTMAT slopes (1,,) 43.07 145 | 163.33 (p =.142) 

Var. in MATHEFF slopes (T22) 63.87 145 | 174.27 (p= .049) 

Var. in USEMATH slopes (733) 28.06 145 | 167.33 (p = .099) 

Var. within schools (o” ) 2697.84 


Results of the random coefficient model show that the average mathematics achievement mean is 444.80 and 
significantly different from zero (.e., p < .001) after including mathematics interest, mathematic self-efficacy, and use of 
ICT in mathematics lessons as level-1 predictors. Mathematics self-efficacy and use of ICT in mathematics lessons are 
significant predictors of the mathematics achievement whereas the mathematic interest is not a significant predictor. 
Mathematics self-efficacy and mathematics achievements have positive significant relationship and one-unit increase in 
mathematics self-efficacy score will result in about 21 unit increase in mathematics achievement. Use of ICT in 
mathematic lessons and mathematics achievements have negative significant relationship and one-unit increase in use of 
ICT in mathematic lessons score will result in about 9 unit decrease in mathematics achievement. Overall, while students 
who have higher mathematics self-efficacy tend to have higher mathematics scores, students tend to have lower 


mathematics scores when the use of ICT in mathematics lessons increases. 


THE EQUATION OF LEVELS 1 & 2 MODELS 


Level-1 Model 
MATHACH, = Boj + Bij * (MATHEFF,;) + Bo; * (USEMATH,;) + 7; 2 
Level-2 Model 
Bo; = Yoo + Yor * (MACTIV,) + Yo2 * (SMRATIO,) + Uo; 3 
By = Yo + Ya * (MACTIV,) + Y12 * (SMRATIO;) + Wy; 4 
Bo = 20 5 
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Within school mathematics score variation is (Oo ° ) 2697.84 which was 3146.83 in the fully unconditional model. 
The addition of three level-1 predictors reduced the 14.3 of the within school variance. The effect mathematics interest and 
use of ICT in mathematics classes on mathematics achievement scores do not significantly differ across schools. Only the 
impact of mathematics self-efficacy on mathematic achievement differed significantly across schools. The exploratory 
analysis results show that mathematics extracurricular activities (MACTIV) at school and math teacher-student ratio 
(SMRATIO) could be potential predictors of the intercept and mathematics self-efficacy which have the only significant 


random coefficients. 


The final model was developed using the results of the random coefficients model and exploratory analysis 
results. Mathematics interest was removed from the model because it is not a significant predictor of the mathematics 
achievement. Also, random coefficient of the use of ICT in mathematics classroom was removed from the model because it 
is not significant. Finally, MACTIV and SMRATIO were added to intercept and mathematics self-efficacy slope 
depending on the suggestion of the exploratory analysis. Therefore, the final model is as follows, and the result of the final 


model is indicated in Table 3. 


Table 3: The Results of the Final Model/Contextual Model 


Fixed Effects Coefficient (SE) t (df) Pp Reliability 

Model for Mean School Math Ach (69) 
Intercept (Vg9) 431.28 (10.82) | 39.85 (147) | <.001 961 
MACTIV (71) 17.88 (4.41) 4.05 (147) | <.001 
SMRATIO (yYo2) -.09 (.03) -2.80 (147) | =.001 
Model for MATHEFF slope (7, ) 
Intercept (710) 23.78 (3.39) 7.00 (147) | <.001 171 
MACTIV (711) -1.12 (1.16) -.96 (132) | .338 
SMRATIO (7,2) -.00 (.01) -.27 (71) 789 
Model for USEMATH slope(f,) 
Intercept (V29) -8.53 (1.06) -8.05(346) | <.001 

Random Effects (Var. Components) | Variance | df XV 5 

Var. in school means (Tg) 4258.82 | 147 | 4639.20 (p < .001) 

Var. in MATHEFF slopes (7,4) 48.56 147 | 192.79 (p = .007) 

Var. within schools (o” ) 2763.83 


Results of the contextual model indicate that the mean mathematics achievement (799) is 431.28 and statistically 
different from zero (p < .001) after controlling other school level variables. The impacts of mathematics extra-curricular 
activities (MACTIV) and student-mathematics teacher ratio (SMRATIO) on mathematics achievement are both 
statistically significant. The effect of MACTIV on mean math achievement is 17.88 which means that the average 
mathematics achievement increases by 17.88 units for one unit increase in MACTIV after controlling other variables. 
Schools with more extracurricular math activities have higher average mathematics achievement. The effect of SMRATIO 
is -.09 and statistically significant which means that the average mathematics achievement decreases by .09 for one-unit 
increase in SMRATIO after controlling other variables. This result indicates that schools with lower student-mathematics 


teacher have higher mathematics achievement than schools with higher student-mathematics achievements. 
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The average effect of the mathematics self-efficacy is 23.78 and statistically significant which means that on 
average students with higher mathematics self-efficacy scores outperform students with lower mathematics self-efficacy 
scores. However, neither MACTIV nor SMRATIO have significant effect on the effect of the mathematics self-efficacy on 
mathematics achievement. The average effect of use of ICT in mathematics classrooms (USEMATH) is negative and 
statistically significant. This result suggests that schools who uses ICT in the mathematics classroom more have lower 


mathematics achievements. 


Between school variation in mean mathematics achievement (To9) was 5288.9 and it decreased to 4258.82 with 
the addition of the MACTIV and SMRATIO to the intercept of the model. Therefore, addition of these two variables 
explained about 19.5% of the between school variance. However, the between school variance is still significant which 
yields addition of more level-2 variables to the intercept. Additionally, even if the impacts of MACTIV and SMRATIO on 
the effect of mathematics self-efficacy on mathematic achievement are not significant, addition of these two variables 
decreased the slope variation from 63.87 to 48.56. However, the slope variation is still significant which yielded addition 


of other level-2 variables to the slope. 
DISCUSSIONS 


According to the results provided, mathematics self-efficacy and is found to be positively correlated with mathematics 
achievement, while ICT usage was negatively correlated with the student level. For school level, an exploratory analysis 
showed that mathematics extracurricular activities at school and math teacher-student ratio could be the predictors of 
mathematics mean mathematics achievement and further analysis approved that these school level variables were 


significant predictors for average mean math achievement. 


Self-efficacy is found to have the greatest positive effect on student achievement. This finding is compatible with 
what Wong (1992) found for self-expectation. The researchers can conclude here that the more students believe that they 
can succeed, the more they will be successful in mathematics. The negative effect of ICT usage on mathematics 
achievement arises the question pf productivity of the computer use. The literature about the effect of computer usage on 
mathematics achievement varies. Some researchers find a positive effect while some find the negative effect. The potential 
reason for the negative effect might be because of the emerged “over-sold, under-used” phenomenon. In other words, 
although almost all of the schools have sufficient computers, they are not used effectively by many teachers (Cuban, 2001) 


which can lead a negative effect. 


All these findings imply that mathematics achievement can be predicted by factors that are not basically cognitive 
and from school level. This finding can be interpreted as there are more than improving instructional quality or better 
curriculum materials for increasing achievement. However, there are some weaknesses of this study that should be noted. 
The exclusion of the missing data from the analysis. So, it can be said that the more data the researcher can include, the 
more precise the results will be. Secondly the limitation of the number of the variables because of time constraints. Under 
all circumstances, an analysis focusing on student affective characteristics and school factors on mathematics achievement 


can be a notable start for understanding the effects of mathematics achievement. 
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